Scene understanding device

ABSTRACT

A scene understanding device obtains map data where one, two or more obstacle detection frames, shaped corresponding to a road structure, for detecting an obstacle are set in advance for a specific spot on a road where a vehicle would otherwise bump into a vehicle or a pedestrian. The scene understanding device determines whether there exists an obstacle in the obstacle detection frames set for the specific spot on a scheduled traveling route of a host vehicle, and calculates the degree of risk at the specific spot based on a result of determining whether there exists an obstacle therein.

TECHNICAL FIELD

The present invention relates to a scene understanding device thatdetermines the degree of risk at a specific spot on a road where avehicle would otherwise bump into another vehicle or a pedestrian.

BACKGROUND

A degree-of-risk calculation apparatus for calculating the degree ofpotential risk around a host vehicle has been proposed (seeInternational Publication No. 2012/032624). According to InternationalPublication No. 2012/032624, based on information from an obstacledetection device, the degree-of-risk calculation apparatus changes themesh setting for gridded areas around the host vehicle, and therebycalculates the risk potentials respectively for the intersections, or inthe gridded areas, in the mesh. Accordingly, based on thethus-calculated risk potentials, the degree-of-risk calculationapparatus sets the target route of the host vehicle.

Since the risk potentials are calculated respectively for all thegridded areas around the host vehicle, a problem arises in which whenmany obstacles are detected, the arithmetic load increases, and it takeslonger to calculate the degree of risk.

SUMMARY

In view of the above problem, the preset invention has been made, and anobject thereof is to provide a scene understanding device which inhibitsan excessive increase in the arithmetic load.

The scene understanding device according to an aspect of the presentinvention determines whether there exists an obstacle in obstacledetection frames which are set in advance for a specific spot where avehicle would otherwise bump into another vehicle or a pedestrian, andwhich are shaped corresponding to a road structure. Thus, based on aresult of the determination, the scene understanding device calculatesthe degree of risk at the specific spot.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an overall configuration of a drivingassistance apparatus 1 a including a scene understanding device of afirst embodiment;

FIG. 2 is a block diagram showing multiple processing circuitsconfigured by an arithmetic circuit 17 a;

FIG. 3 is a flowchart showing an example of a driving assistance methodincluding a scene understanding method of the first embodiment;

FIG. 4(a) is a plan view showing an example of curb information on wherecurbs 41 are in a three-way junction or road network information on thethree-way junction.

FIG. 4(b) is a plan view showing examples of an obstacle detection frame42 to be set for the three-way junction;

FIG. 5(a) is a plan view showing an example of a junction;

FIG. 5(b) is a plan view showing examples of the obstacle detectionframe 42 to be set for the junction;

FIG. 5(c) is a plan view showing an example of a pedestrian crossing 45;

FIG. 5(d) is a plan view showing examples of the obstacle detectionframe 42 to be set for the pedestrian crossing 45;

FIG. 6(a) is a plan view showing examples of a close observation frame48 to be set for a three-way junction with traffic lights;

FIG. 6(b) is a plan view showing examples of the close observation frame48 to be set for a three-way junction without traffic lights;

FIG. 6(c) is a plan view showing how a blind spot caused by an obstacle49 and a close observation frame 48 overlap each other;

FIG. 7(a) is a plan view showing examples of the close observation frame48 to be set for a junction;

FIG. 7(b) is a plan view showing examples of close observation frame 48to be set for a pedestrian crossing;

FIG. 8(a) is a plan view showing a three-way junction for which 11obstacle detection frames (R01 to R11) and two close observation frames(T01, T02) are set;

FIGS. 8(b) to 8(e) are plan views showing examples of how to combine theobstacle detection frames (R01 to R11) including obstacles with theclose observation frames (T01, T02) overlapping blind spots for thethree-way junction shown in FIG. 8(a);

FIGS. 9(a) to 9(d) are plan views showing examples of an obstacledetection frame 42′ to be selected by a detection frame selector 25;

FIGS. 9(e) and 9(f) are plan views showing examples of a closeobservation frame 48′ to be selected by a detection frame selector 25;

FIG. 10 is a block diagram showing an overall configuration of a drivingassistance apparatus 1 b including a scene understanding device of asecond embodiment;

FIG. 11 is a block diagram showing multiple processing circuitsconfigured by an arithmetic circuit 17 b;

FIG. 12 is a flowchart showing an example of a driving assistance methodincluding a scene understanding method of the second embodiment;

FIG. 13 is a flowchart showing a detailed procedure for steps S33 andS23 in FIG. 12;

FIG. 14 is a flowchart showing a detailed procedure for a knowledge tree(for an entrance to a junction) shown for step S 47 in FIG. 13;

FIG. 15 is a flowchart showing a detailed procedure for a knowledge tree(for the inside of a junction) shown for step S 49 in FIG. 13; and

FIG. 16 is a flowchart showing a detailed procedure for a knowledge tree(for an exit from a junction) shown for step S 51 in FIG. 13.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Next, referring to the drawings, detailed descriptions will be providedfor embodiments of the present invention.

First Embodiment

Referring to FIG. 1, descriptions will be provided for an overallconfiguration of a driving assistance apparatus 1 a including a sceneunderstanding device in the first embodiment. The driving assistanceapparatus 1 a is an apparatus which performs driving assistance bydetermining how to run a host vehicle (a driving assistance method)based on a degree of risk (a scene) of collision between the hostvehicle and another vehicle or a pedestrian at a specific spot on itsscheduled traveling route. The scene understanding device is a devicewhich calculates the degree of risk, or understands the scene. Thespecific spot is a spot on a road where a vehicle would otherwise bumpinto another vehicle or a pedestrian. Examples of the specific spotinclude an intersection where three or more roads meet, an interchangeof an expressway, and a pedestrian crossing. Accordingly, when the hostvehicle runs at the specific spot, the scene understanding devicedetects other vehicles or pedestrians at and around the specific spot,and calculates a risk of collision between the host vehicle and anothervehicle or a pedestrian. Thus, in order to enable the host vehicle torun through the specific spot safely, the driving assistance apparatus 1a determines how to run the host vehicle (the driving assistance method)based on the degree of risk, and thereby performs the drivingassistance.

The driving assistance apparatus 1 a includes a GPS 11, a map database12, a vehicle-mounted camera 13, a laser sensor 14, an operation unit15, a degree-of-risk database 16, and an arithmetic circuit 17 a. TheGPS 11 is an example of a vehicle position detector that detects acurrent position of the host vehicle. The map database 12 is an exampleof a map storage for storing map data. The vehicle-mounted camera 13 andthe laser sensor 14 are examples of an obstacle detector that detectspositions of obstacles present around the vehicle. The operation unit 15is a member for receiving instructions from the driver of the hostvehicle, and includes a microphone, a touch panel arranged on aninstrument panel, and a steering switch. The degree-of-risk database 16stores relationships between combinations of obstacle detection framesincluding obstacles and degrees of risk. The degree-of-risk database 16and obstacle detection frames will be described later.

The arithmetic circuit 17 a is a circuit which performs a series ofarithmetic processes for: calculating a degree of risk using obstacleinformation and map information; and thereby performing drivingassistance. The arithmetic circuit 17 a is, for example, ageneral-purpose microcomputer including a CPU, a RAM, a ROM, a memoryand an input/output control circuit. A computer program in which theseries of arithmetic processes is described is installed in themicrocomputer in advance. Executing the computer program, themicrocomputer constructs multiple processing circuits for executing theabove-mentioned series of arithmetic processes. The multiple processingcircuits constructed by the arithmetic circuit 17 a are described laterby reference to FIG. 2.

The GPS 11 measures the position of the host vehicle in real time byreceiving electric waves from the NAVSTAR satellites in the GlobalPositioning System. For each specific spot, one, two or more obstacledetection frames, shaped corresponding to the structure of a road, fordetecting obstacles are set in advance in the map data stored in the mapdatabase 12. The vehicle-mounted camera 13 is mounted on the hostvehicle, and obtains ambient images by shooting the surroundings of thehost vehicle. The arithmetic circuit 17 a analyzes the ambient images,and thereby determines whether there exists an obstacle around the hostvehicle, and where an obstacle, if any, is located. The laser sensor 14emits pulses of laser light, detects light reflected from the obstacle,thereby detecting the distance from the host vehicle to the obstacle,and the azimuth of the obstacle relative to the host vehicle.

Referring to FIG. 2, descriptions will be provided for the multipleprocessing circuits constructed by the arithmetic circuit 17 a. As themultiple processing circuits, the arithmetic circuit 17 a includes ascene understanding unit 21 for determining a driving assistance methodby the calculating the degree of risk, and a driving assistance unit 22for performing the determined driving assistance. The sceneunderstanding unit 21 includes a map obtaining unit 23, a routecalculator 24, a detection frame selector 25, an obstacle determinationunit 26, a degree-of-risk calculator 27, a blind spot overlapdetermination unit 28, an encoding processor 29, and a degree-of-riskdata obtaining unit 30.

The driving assistance unit 22 performs the driving assistance inaccordance with how to run the host vehicle (the driving assistancemethod), which is determined by the scene understanding unit 21. To putit specifically, the driving assistance may be an autonomous drivingcontrol in which the driving assistance apparatus 1 a autonomouslyperforms all the driving operation, including steering operation andpedal operation, by driving various actuators. Otherwise, the drivingassistance may be one in which through the driver's five senses such ashearing, sight and touch, the driving assistance apparatus 1 a gives thedriver instructions, suggestions or hints as to how the driver shouldperform driving operation.

The route calculator 24 calculates a scheduled traveling route from thecurrent position of the host vehicle measured by the GPS 11 to adestination received by the operation unit 15. Incidentally, theembodiment will describe a case where the driving assistance apparatus 1a or the scene understanding device has a function of autonomously doingthe arithmetic on the scheduled traveling route. However, the drivingassistance apparatus 1 a or the scene understanding device may obtain ascheduled traveling route calculated by a difference device from theoutside.

The map obtaining unit 23 obtains map data on the scheduled travelingroute from the map database 12. The map obtaining unit 23 reads specificspots on the scheduled traveling route, and the obstacle detectionframes set for each specific spot. A digital map may be used as the mapdata. The digital map includes curb information on where a curb 41 islocated, or road network information shown in FIG. 4(a). The curbinformation is used to calculate a travelable area of the host vehicle.The road network information is used to obtain an area where the hostvehicle will be able to travel at the next point of time. The digitalmap further includes information on obstacle detection frames which areshaped corresponding to the structure of the road.

Although the embodiment shows the case where the map database 12 isprovided onboard the vehicle, the map database 12 is not limited tothis. The map database 12 may be stored in a server outside the vehicle.In this case, the map obtaining unit 23 may obtain the map data from theoutside of the vehicle via a network. This is also the case with thedegree-of-risk database 16. Furthermore, the obstacle detection framesmay be set on the map database 12 from the beginning.

As shown in FIG. 4(b), multiple obstacle detection frames 42 shapedcorresponding to the structure of the road are set for one specificpoint (for example, a three-way junction). The obstacle detection frames42 are set for the entrances to and the exits from the intersection, theinside of the intersection, the pedestrian crossings, the sidewalkportions connected adjacent to the pedestrian crossings, and the like.Inside the intersection, the obstacle detection frames 42 are set forthe straight-ahead, right-turn and left-turn routes passing through theintersection.

Another example of a specific spot where a vehicle would otherwise bumpinto another vehicle is a merging point where, as shown in FIGS. 5(a)and 5(b), a merging lane 43 (including a branch lane) merges into athrough traffic lane 44. An example of a specific spot where a vehiclewould otherwise bump into a pedestrian is a pedestrian crossing 45, asshown in FIGS. 5(c) and 5(d). In these specific spots, too, multipleobstacle detection frames 42 are set for the pre-encounter traffic(scene entrance), the encounter traffic (scene inside), and thepost-encounter (scene exit).

The obstacle determination unit 26 determines whether there exists anobstacle in the obstacle detection frames set for the specific spot onthe scheduled traveling route. The obstacle determination unit 26determines whether the location of an obstacle detected by thevehicle-mounted camera 13 and the laser sensor 14 falls within theobstacle detection frames.

The degree-of-risk calculator 27 calculates the degree of risk at thespecific spot based on a result of determining whether there exists anobstacle. A specific method of calculating the degree of risk will bedescribed later, but is not limited to descriptions which will beprovided for the method later. An already-known method (for example, amethod recited in Patent Literature 1) may be used depending on thenecessity.

From the curb information or the road network information, the sceneunderstanding unit 21 obtains a running area where the host vehicle willrun from now. In a case where a specific spot is included in the runningarea, the scene understanding unit 21 reads the obstacle detectionframes set for the specific spot. An obstacle is detected using theexternal sensing devices (the vehicle-mounted camera 13 and the latersensor 14) mounted on the vehicle. The scene understanding unit 21determines whether the detected obstacle is included in the obstacledetection frames. In a case where the obstacle exists in thepredetermined obstacle detection frames set for the specific spot, thescene understanding unit 21 determines that the specific spot isdangerous. The degree of risk may be set for each obstacle detectionframe, and may be set differently for each obstacle detection frame. Inother words, when the degree of risk is calculated, the degree of riskis differently weighted for each obstacle detection frame.

As described above, the scene understanding unit 21 calculates thedegree of risk at the specific spot based on whether there exists anobstacle in the obstacle detection frames 42 set in advance in the mapdata. Thereby, obstacles which are detected at positions having nothingto do with the calculation of the degree of risk can be excluded fromwhat the scene understanding unit 21 processes. Accordingly, it ispossible to inhibit an excessive increase on the arithmetic load.

In the map data stored in the map database 12, not only the obstacledetection frames, but also close observation frames to be closelyobserved from a viewpoint of whether the close observation framesoverlap blind spots caused by obstacles may be set in advance for eachspecific spot. In this case, the map obtaining unit 23 obtains the mapdata where the close observation frames are set in advance for thespecific spot. As shown in FIG. 2, the scene understanding unit 21further includes the blind spot overlap determination unit 28 thatdetermines whether blind spots caused by obstacles overlap the closeobservation frames. Based on a result of determining whether blind spotscaused by obstacles overlap the close observation frames, thedegree-of-risk calculator 27 calculates the degree of risk at eachspecific spot. This makes it possible to calculate the degree of risk atthe specific spot on the assumption that there exist obstacles in blindspots.

A close observation frame 48 is set for a place where a blind spot islikely to occur due to the existence of another vehicle, a building or awall. Furthermore, a close observation frame 48 is set for a place whichwill be dangerous when another vehicle or a pedestrian comes out of ablind spot. A place with which to provide a close observation frame 48varies depending on the route of the host vehicle 46 and the directionin which the host vehicle 46 approaches the specific spot. Even in acase where the host vehicle approaches the same specific spot aftertravelling the same route, places with which to provide a closeobservation frame and the number of close observation frames may varydepending on cases. For example, the number of close observation framesneeded for a specific spot varies depending on whether there are trafficlights in the specific spot.

FIG. 6(a) shows examples of the close observation frame 48 to be set fora three-way junction with traffic lights. When there exists anothervehicle at the entrance to the opposing lane, the bike lane at the sideof the vehicle is likely to fall into a blind spot. When a vehiclerunning ahead is inside the intersection, a blind spot is likely tooccur on and near the pedestrian crossing, as well as at and near theexit from the lane on which the host vehicle is running. For thesereason, the close observation frame 48 is set for these places which arelikely to fall into a blind spot.

FIG. 6(b) shows examples of the close observation frame 48 to be set fora three-way junction without traffic lights. A scene in which anothervehicle enters the intersection from another road crossing the road onwhich the host vehicle 46 is running needs to be taken intoconsideration to calculate the degree of risk. When a vehicle runningahead is inside the intersection, a blind spot is likely to occur on andnear the pedestrian crossing, as well as at and near the exit from. Ablind spot is likely to occur at the entrance to the opposing lane inanother road due to a vehicle running ahead. For this reason, the closeobservation frame 48 is set for the entrance to the opposing lane inanother road.

The degree-of-risk calculator 27 calculates the degree of risk from howmuch of a blind spot 50 obtained by the sensor attached to the hostvehicle 46 overlaps a close observation frame 48. For example, thedegree-of-risk calculator 27 calculates the degree of risk from aproportion of the area where the blind spot and the close observationframe 48 overlap each other to the area of the close observation frame48. Otherwise, the degree-of-risk calculator 27 may calculate the degreeof risk from a proportion of the length 48 a of the close observationframe 48 overlapping the blind spot to the outer circumference of theclose observation frame 48. The degree-of-risk calculator 27 is capableof calculating a higher degree of risk when a value representing howmuch of the blind spot 50 obtained by the sensor attached to the hostvehicle 46 overlaps the close observation frame 48 is greater than areference value, because the higher value means a worse visibility.

In a case where as shown in FIG. 7(a), there is a wall between a merginglane 43 and a through-traffic lane 44 at a merging point, the areabehind the wall is likely to fall into a blind spot. Furthermore, in acase where a vehicle running ahead is changing lanes from the merginglane 43 to the through-traffic lane 44, a blind spot is likely to occurin and near an area beyond a place where the vehicle merges into thethrough-traffic lane 44 are likely to fall into a blind spot. For thesereasons, the close observation frame 48 is set for these places whichare likely to fall in a blind spot. Incidentally, when the host vehicle46 enters the through-traffic lane 44 from the merging lane 43, multipleclose observation frames 48 are set. However, in a case where the hostvehicle 46 only runs straight ahead in the through-traffic lane 44 towhich the right of way is given over the merging lane 43, no closeobservation frame is provided.

In a case where as shown in FIG. 7(b), a vehicle running ahead is on apedestrian crossing, a place at the side of the pedestrian crossing islikely to fall into a blind spot. Furthermore, in a case where apedestrian is walking on the pedestrian crossing, a place behind thepedestrian is likely to fall into a blind spot. For these reasons, theclose observation frame 48 is provided for these places which are likelyto fall into a blind spot.

In the first embodiment, based on combinations of the obstacle detectionframes 42 including obstacles, the degree-of-risk calculator 27calculates the degree of risk at the specific spot. Since thedegree-of-risk calculator 27 need not calculate the degree of risk foreach obstacle detection frame 42, it is possible to inhibit an excessiveincrease in the arithmetic load. Furthermore, the degree-of-riskcalculator 27 may calculate the degree of risk at the specific spot byadding the close observation frames 48 overlapping the blind spotscaused by the obstacles to the combinations of the obstacle detectionframes 42.

As discussed above, multiple obstacle detection frames 42 and multipleclose observation frames 48 are set for one specific spot. Thedegree-of-risk calculator 27 determines whether a traffic condition setin advance can be read from the combinations of the multiple obstacledetection frames 42 from which the obstacles are detected and themultiple close observation frames 48 overlapping the blind spots. Onlyif the traffic condition set in advance can be read, the degree-of-riskcalculator 27 calculates the degree of risk based on the trafficcondition. In a case where the traffic condition set in advance can beread while the host vehicle is running, the degree of risk calculator 27recognizes the environment to be encountered by the host vehicle asbeing a dangerous scene.

In this respect, the degree of risk is determined using the combinationsof the obstacle detection frames 42 including the obstacles and theclose observation frames 48 overlapping the blind spots. Nevertheless,the degree of risk may be determined by: calculating the degrees of riskfor the respective obstacle detection frames 42 and the degrees of riskfor the respective close observation frames 48; and adding up thesedegrees of risk.

FIG. 8(a) shows a three-way junction for which 11 obstacle detectionframes (R01 to R11) and two close observation frames (T01, T02) are set.As a scheduled traveling route 51, the host vehicle 46 turns left at thethree-way junction. FIGS. 8(b) to 8(e) show examples of how to combinethe obstacle detection frames 42 including the obstacles with the closeobservation frames 48 overlapping the blind spots for the three-wayjunction shown in FIG. 8(a). The obstacle detection frames 42 includingthe obstacles and the close observation frames 48 overlapping the blindspots are hatched. In an example of FIG. 8(b), obstacles are detected inobstacle detection frames (R04, R06) set for: the entrance to theintersection from the opposing lane and its vicinity; and the exit fromthe intersection to the opposing lane and its vicinity. From acombination of the obstacle detection frames (R04, R06), thedegree-of-risk calculator 27 can read the traffic condition as trafficbeing congested on the opposing lane. In an example of FIG. 8(c),obstacles are detected in obstacle detection frames (R02, R05, R07) setfor the inside of the intersection, as well the exit from theintersection for the host vehicle and its vicinity. From a combinationof the obstacle detection frames (R02, R05, R07), the degree-of-riskcalculator 27 can read the traffic condition as traffic being congestedwith vehicles waiting in the intersection to turn right because othervehicles stay on the lane to which they are going to turn right.

In an example of FIG. 8(d), an obstacle is detected in an obstacledetection frame (R05) set for the inside of the intersection in front ofthe pedestrian crossing. In addition, a close observation frame (T02) onand near the pedestrian crossing at the exit from the intersection forthe host vehicle overlaps a blind spot. From a combination of theobstacle detection frame (R05) and the close observation frame (T02),the degree-of-risk calculator 27 can read the traffic condition asanother vehicle stopping in front of the pedestrian crossing becausethere are pedestrians 53 on the pedestrian crossing, or the pedestriancrossing being invisible because the obstacle exists in front of thepedestrian crossing.

In an example of FIG. 8(e), an obstacle is detected in an obstacledetection frame (R05) set for the inside of the intersection on theopposing lane. In addition, a close observation frame (T01) set for anentrance to the intersection from the opposing lane overlaps a blindspot. From a combination of the obstacle detection frame (R05) and theclose observation frame (T01), the degree-of-risk calculator 27 can readthe traffic condition as the side road at the side of the entrance tothe intersection from the opposing lane being invisible because anothervehicle exists inside the intersection on the opposing lane. Thereby,the degree-of-risk calculator 27 can determine a risk that a bike 52 mayexist in the side road at the side of the entrance to the intersectionfrom the opposing lane.

As shown in FIG. 1, the scene understanding device of the firstembodiment includes the degree-of-risk database 16. The combinations ofthe obstacle detection frames 42 including the obstacles are encoded.The degree-of-risk database 16 stores relationships between the encodedcombinations and the degrees of risk. The scene understanding unit 21includes the encoding processor 29 that encodes the combinations of theobstacle detection frames including the obstacles. Using thedegree-of-risk database 16, the degree-of-risk calculator 27 calculatesthe degree of risk at the specific spot from the combinations encoded bythe encoding processor 29. The encoding makes it possible to inhibit theincrease in the arithmetic load more. Incidentally, it is a matter ofcourse that the close observation frames 48 overlapping the blind spotsmay be added to the encoded combinations.

The encoding is a method of representing information on the degrees ofrisk which is based on a high-speed information process to be performedby a computer using bit strings. Results of scene understanding usingthe multiple obstacle detection frames and the multiple closeobservation frames are used for the encoding. How to associate thecombinations with the degrees of risk is based on past accident cases,and past incident cases (near-accident cases which would have naturallyresulted in major disasters or accidents although actually not). Thedegree-of-risk database 16 stores the past accident cases as digitizedusing the encoding technique.

For example, each combination of the obstacle detection frames 42including the obstacles with the close observation frames 48 overlappingthe blind spot is represented by a series of digits. The combinationsshown in FIGS. 8(b) to 8(e) are encoded and associated with thecorresponding degrees of risk, and are stored in the degree-of-riskdatabase 16. The result of encoding the combination shown in FIG. 8(b)is “0001010000000.” The result of encoding the combination shown in FIG.8(c) is “010010001000.” The result of encoding the combination shown inFIG. 8(d) is “0000100100001. The result of encoding the combinationshown in FIG. 8(e) is “0000100100010.”

The degree-of-risk calculator 27 compares the combinations of theobstacle detection frames and the close observation frames, which areencoded by the encoding processor 29, with the encoded combinationsstored in the degree-of-risk database 16, and thereby calculates thedegree of risk which corresponds to the combinations of the obstacledetection frames and the close observation frames.

Furthermore, in order to expand the scope of what can be represented bydigits in the encoding, not only whether there exist obstacles and blindspots, but also attribute information on obstacles themselves may berepresented by digits. The obstacle determination unit 26 may beconfigured to detect the attributes of the obstacles existing in theobstacle detection frames 42 set for the specific spot on the scheduledtraveling route 51. The encoding processor 29 encodes the combinationsof the obstacle detection frames 42 including the obstacles and theattributes of the obstacles. Since the attribute information on theobstacles is taken into consideration, the accuracy of the calculationof the degrees of risk is improved. It is a matter of course that theclose observation frames 48 overlapping the blind spots may be added tothese combinations.

As a method of representing the attribute of each obstacle using digits,bit strings representing the combinations may be increased in numbers byencoding the attribute thereof with the binary-bit representation using0 and 1. The attribute information includes physical information andcharacteristic information. Examples of the physical informationinclude: information on vehicle specifications including the weights andsizes of vehicles; and information on types of obstacles (a pedestrian,a bicycle and a four-wheeler). Examples of the characteristicinformation include: information on whether each obstacle is static orin motion; and motion information such as on how each obstacle, if inmotion, is moving.

The first embodiment has shown the case where, as shown in FIGS. 4 to 7,the degree of risk is calculated using all of the obstacle detectionframes 42 and the close observation frames 48 set in advance for eachspecific spot. However, the embodiment is not limited to this. Forexample, the embodiment may be such that: obstacle detection frames 42′are selected from the obstacle detection frames 42 set in advance forthe specific spot; and it is determined whether there exists an obstaclein the selected obstacle detection frames 42′.

As shown in FIG. 2, the scene understanding unit 21 further includes thedetection frame selector 25 that selects obstacle detection frames 42′to be determined depending on the scheduled traveling route 51 from theobstacle detection frames 42′ set in advance for each specific spot. Theobstacle determination unit 26 determines whether there exists anobstacle in the obstacle detection frames 42′ selected by the detectionframe selector 25. This makes it possible to inhibit an increase in thearithmetic load on the obstacle determination unit 26.

Furthermore, the detection frame selector 25 may select closeobservation frames 48′ to be determined depending on the scheduledtraveling route 51 from the close observation frames 48 set in advancefor each specific spot. In this case, the blind spot overlapdetermination unit 28 may determine whether a blind sport caused by anobstacle overlaps the close observation frames 48′ selected by thedetection frame selector 25. This makes it possible to inhibit anincrease in the arithmetic load on the blind spot overlap determinationunit 28.

FIGS. 9(a) to 9(d) show examples of the obstacle detection frame 42′ tobe selected by the detection frame selector 25. FIG. 9(a) shows examplesof the obstacle detection frame 42′ selected for a three-way junctionwith traffic lights. FIG. 9(b) shows examples of the obstacle detectionframe 42′ selected for a three-way junction without traffic lights. FIG.9(c) shows examples of the obstacle detection frame 42′ selected for amerging point. FIG. 9(d) shows examples of the obstacle detection frame42′ selected for a pedestrian crossing. FIGS. 9(e) and 9(f) showexamples of the close observation frame 48′ to be selected by thedetection frame selector 25. FIG. 9(e) shows examples of the closeobservation frame 48′ selected for a four-way junction with trafficlights. FIG. 9(f) shows examples of the close observation frame 48′selected for a four-way junction without traffic lights.

The selection method will be described using a three-way junction as anexample. First of all, the obstacle detection frame 42′ on the scheduledtraveling route 51 of the host vehicle 46 is selected. The obstacledetection frame 42′ on the opposing lane crossing the scheduledtraveling route 51 of the host vehicle 46 is selected. In addition, theclose observation frames 48′ in contact with the selected obstacledetection frames 42′ are selected. Thereby, the obstacle detectionframes 42′ and the close observation frames 48′ related to the movementof the host vehicle 46 can be selected. The above selection method isalso applicable to the other specific spots such as a merging point anda pedestrian crossing.

Referring to FIG. 3, descriptions will be provided for examples of thescene understanding method and the driving assistance method using thedriving assistance apparatus 1 a including the scene understandingdevice of the first embodiment.

In step S01, the map obtaining unit 23 obtains the map data where one,two or more obstacle detection frames 42 for detecting an obstacle areset in advance for the specific spots. Incidentally, as for the timingof reading the obstacle detection frames 42, the configuration may besuch that each time the vehicle approaches a specific spot, the mapobtaining unit 23 reads the obstacle detection frames 42 set for thespecific spot which the vehicle is approaching. Proceeding to step S03,the route calculator 24 calculates the scheduled traveling route 51 ofthe host vehicle 46 based on information on the position and destinationof the host vehicle 46. In step S05, the obstacle determination unit 26obtains information on the obstacles around the vehicle which aredetected by the vehicle-mounted camera 13 and the laser sensor 14. Instep S07, the obstacle determination unit 26 obtains information on theattributes of the obstacles which are detected by the vehicle-mountedcamera 13 and the laser sensor 14.

Proceeding to step S11, the blind spot overlap determination unit 28calculates the blind spot ranges caused by the obstacles which aredetected by the vehicle-mounted camera 13 and the laser sensor 14.Proceeding to step S13, the scene understanding unit 21 determineswhether the nearest specific spot on the scheduled traveling route 51 isan intersection where three or more roads meet. Descriptions will beprovided for a procedure for how to determine the specific spot as anintersection. A similar procedure is applicable to the other specificspots.

Proceeding to step S15, the detection frame selector 25 selects theobstacle detection frames 42′ and the close observation frames 48′ to bedetermined depending on the scheduled traveling route 51 from theobstacle detection frames 42 and the close observation frames 48 set inadvance for the intersection. Proceeding to step S17, the blind spotoverlap determination unit 28 determines whether the blind spots causedby the obstacles overlap the close observation frames 48′. Proceeding tostep S19, the encoding processor 29 encodes the combinations of theobstacle detection frames 42 including the obstacles. Thereafter, thedegree-of-risk calculator 27 reads data on the relationships between theencoded combinations and the degrees of risk from the degree-of-riskdatabase 16.

Proceeding to step S21, the degree-of-risk calculator 27 compares thecombinations encoded by the encoding processor 29 with the data on therelationships between the encoded combinations and the degrees of risk,and thereby calculates the degree of risk for the specific spot. In stepS23, the degree-of-risk calculator 27 determines the driving assistancemethod depending on the calculated degree of risk, and outputs thedetermined driving assistance method to the driving assistance unit 22.Proceeding to step S25, the driving assistance unit 22 performs thedriving assistance in accordance with the determined assistance method.

As discussed above, the following operation and effects can be obtainedfrom the first embodiment of the present invention.

The scene understanding device calculates the degree of risk at eachspecific spot based on whether there exists an obstacle in the obstacledetection frames 42 which are set in the map data in advance, and whichare shaped corresponding to the road structure. Thereby, obstacles whichare detected at positions having nothing to do with the calculation ofthe degree of risk can be excluded from what the scene understandingdevice needs to process. This inhibits an excessive increase in thearithmetic load.

As shown in FIG. 6, the degree-of-risk calculator 27 calculates thedegree of risk at each specific spot based on whether the closeobservation frames 48 to be closely observed by the host vehicle 46overlap the blind spots 50 caused by the obstacles 49. This makes itpossible to calculate the degree of risk at the specific spot on theassumption that obstacles exist in the blind spots 50.

The degree-of-risk calculator 27 calculates the degree of risk at eachspecific spot based on the combinations of the obstacle detection framesincluding the obstacles. Thus, the degree-of-risk calculator 27 need notcalculate the degree of risk for each obstacle detection frame 42. Thismakes it possible to inhibit an excessive increase in the arithmeticload.

Using the degree-of-risk database 16, the degree-of-risk calculator 27calculates the degree of risk at each specific spot from the encodedcombinations of the obstacle detection frames. The encoding makes itpossible to inhibit the increase in the arithmetic load more.

The obstacle determination unit 26 detects the attributes of theobstacles in the obstacle detection frames 42 at each specific spot onthe scheduled traveling route 51, and the encoding processor 29 encodesthe combinations of the obstacle detection frames including theobstacles and the attributes of the obstacles. Since the attributes(physical information and characteristic information) of the obstaclesare taken into consideration, the accuracy of the calculation of thedegree of risk is improved.

As shown in FIG. 9, the detection frame selector 25 selects the obstacledetection frames 42′ to be determined depending on the scheduledtraveling route 51 from the obstacle detection frames 42 set in advancefor each specific spot. The obstacle determination unit 26 determineswhether an obstacle exists in the obstacle detection frames 42′ selectedby the detection frame selector 25. Since the obstacle determinationunit 26 makes the determination on only the obstacle detection frames42′ selected by the detection frame selector 25, it is possible toinhibit the increase in the arithmetic load more.

In the case where the specific spot is an intersection where three ormore roads meet, the obstacle detection frame 42 is set for the entranceto, and the exit from, the intersection, the inside of the intersection,and the pedestrian crossings. This makes it possible to inhibit anexcessive increase in the arithmetic road when the degree of risk iscalculated for an intersection where three or more roads meet.

Second Embodiment

Referring to FIGS. 10 and 11, descriptions will be provided for aconfiguration of a driving assistance apparatus 1 b including a sceneunderstanding device of the second embodiment. The driving assistanceapparatus 1 b is different from the driving assistance apparatus shownin FIG. 1 in that the driving assistance apparatus 1 b includes aknowledge database 17 instead of the degree-of-risk database 16. Theknowledge database 17 stores data (knowledge tree) on: the obstacledetection frames 42 to be determined depending on the position of thehost vehicle at each specific spot; and the order of the obstacledetection frames 42 to be cautious about. Examples of the position ofthe host vehicle at the specific spot include the entrance to, theinside of, and the exit from, the specific spot. The obstacle detectionframes 42, and the order of the obstacle detection frames 42 to becautious about are set for each of the entrance to, the inside of, andthe exit from, the specific spot. It is a matter of course that theclose observation frames 48 and the order of the close observationframes 48 to be cautious about may be set depending on the position ofthe host vehicle.

Referring to FIG. 11, descriptions will be provided for the multiplearithmetic processors configured by an arithmetic circuit 17 b. Thearithmetic circuit 17 b is different from the arithmetic circuit 17 ashown in FIG. 2 in that the arithmetic circuit 17 b includes a knowledgetree obtaining unit 31 instead of the encoding processor 29 and thedegree-of-risk data obtaining unit 30. The rest of the configuration ofthe arithmetic circuit 17 b is the same as that of the arithmeticcircuit 17 a. The knowledge tree obtaining unit 31 obtains the data(knowledge tree) on the obstacle detection frames 42 and the order ofthe obstacle detection frames 42 to be cautious about, which areassociated with the position of the host vehicle detected by the GPS 11,from the knowledge database 17. Based on the obstacle detection frames42 and the order of the obstacle detection frames 42 to be cautiousabout which are obtained from the knowledge database 17, the obstacledetermination unit 26 determines whether there exists an obstacle in theobstacle detection frames 42 sequentially. Thereby, depending on theposition of the host vehicle at the specific spot, an appropriate degreeof risk and an appropriate driving assistance method can be calculated.

Using an intersection as an example, descriptions will be provided for amethod of calculating a degree of risk (a driving assistance method)using the knowledge tree. Areas (obstacle detection frames 42 and closeobservation frames 48) to be cautious about at each intersection, andthe order of care taken to the multiple areas are set in the knowledgetree. The knowledge tree includes, for example, “intersection entranceinformation,” “intersection exit information,” “intersection insideinformation,” and “blind spot information.”

To put it specifically, the “intersection entrance information” isinformation on whether there exists another vehicle at or near theentrance to an intersection. The “intersection exit information” isinformation on whether there exists another vehicle at or near the exitfrom an intersection. The “intersection inside information” isinformation on whether there exists another vehicle inside theintersection. The “blind spot information” is information on whether ablind spot caused by another vehicle inside the intersection hides aclose observation frame 48.

These sets of information are obtained in a predetermined order todetermine a type of behavior of the vehicle, that is to say, the drivingassistance method. Types of behavior includes “stop at the stop line” atthe entrance to the intersection, “stop in the right-turn waiting area,”“stop in front of the pedestrian crossing,” “move at low speed to aplace with better visibility, and accelerate or stop,” and “go throughthe intersection within a vehicle speed limit.” The use of the knowledgetree makes it possible to determine one from the speed controlassistance methods.

The knowledge tree differs depending on the position of the host vehicleat each specific spot. Different knowledge trees are preparedrespectively for a case where the host vehicle is in front of theentrance to the intersection, a case where the host vehicle is betweenthe entrance to the intersection and the right-turn waiting area, and acase where the host vehicle is between the right-turn waiting area andthe pedestrian crossing. These knowledge trees are stored in theknowledge database 17.

Using FIG. 12, descriptions will be provided for a scene understandingmethod and a driving assistance method using the driving assistanceapparatus 1 b including the scene understanding device of the secondembodiment. Steps in FIG. 12 which are the same as those in FIG. 3 willbe denoted by the same reference signs. Descriptions for such steps willbe omitted.

The flowchart in FIG. 12 is different from that in FIG. 3 in that theflowchart in FIG. 12 includes step S31 (knowledge tree acquisition) andstep S33 (degree of risk calculation) instead of step S19 (encoding, anddegree-of-risk data acquisition) and step S21 (degree of riskcalculation). The other steps in the flowchart in FIG. 12 are the sameas those in the flowchart in FIG. 3.

In step S31, the knowledge tree obtaining unit 31 obtains the data(knowledge tree) on the obstacle detection frames 42, the closeobservation frames 48, as well as the order of the obstacle detectionframes 42 and the close observation frames 48 to be cautious about,which are associated with the position of the host vehicle detected bythe GPS 11, from the knowledge database 17.

Proceeding to step S33, the obstacle determination unit 26 determineswhether there exists an obstacle in the obstacle detection frames 42sequentially based on the knowledge tree obtained from the knowledgedatabase 17. The degree-of-risk calculator 27 calculates the degree ofrisk at the specific spot, depending on whether there exists anobstacle. Proceeding to step S23, the degree-of-risk calculator 27determines the driving assistance method corresponding to the calculateddegree of risk, and outputs the determined driving assistance method tothe driving assistance unit 22.

Referring to FIG. 13, detailed descriptions will be provided for stepsS33 and S23. In step S41, the obstacle determination unit 26 determineswhether the position of the host vehicle detected by the GPS 11 islocated in front of the entrance to the intersection. If the positionthereof is located in front of the entrance to the intersection (YES instep S41), the process proceeds to step S47, where the knowledge tree(intersection entrance) associated with the entrance to the intersectionis performed to calculate the degree of risk, and thereby to determinesthe driving assistance method. The details of the knowledge tree(intersection entrance) will be described later by referring to FIG. 14.

If the position of the host vehicle is not located in front of theentrance to the intersection (NO in step S41), the process proceeds tostep S43, where it is determined whether the position of the hostvehicle is located between the entrance to the intersection and theright-turn waiting area. If the determination is affirmative (YES instep S43), the process proceeds to step S49, where the knowledge tree(intersection inside) associated with the inside of the intersection isperformed to calculate the degree of risk, and thereby to determines thedriving assistance method. The details of the knowledge tree(intersection inside) will be described later by referring to FIG. 15.

If the determination is negative (NO in step S43), the process proceedsto step S45, where it is determined whether the position of the hostvehicle is located between the right-turn waiting area and the front ofthe pedestrian crossing. If the determination is affirmative (YES instep S45), the process proceeds to step S51, where the knowledge tree(intersection exit) associated with the exit from the intersection isperformed to calculate the degree of risk, and thereby to determine thedriving assistance method. The details of the knowledge tree(intersection exit) will be described later by referring to FIG. 16.

Referring to FIG. 14, descriptions will be provided for the detailedprocedure for the knowledge tree (intersection entrance) shown for stepS47 in FIG. 13. It is determined whether there exists another vehicle ator near the entrance to the intersection based on the above-mentioned“intersection entrance information D01.” If another vehicle existsthere, to stop at the stop line is determined as the behavior of thehost vehicle (S71). If no other vehicle exists there, it is determinedwhether there exists another vehicle at or near the exit from theintersection based on the “intersection exit information D03.” Ifanother vehicle exists there, to stop at the stop line is determined asthe behavior of the host vehicle (S73). If no other vehicle existsthere, it is determined whether there exists another vehicle stoppinginside the intersection based on “vehicle-stopping-intersection-insideinformation D05.” If no other vehicle exists there, to move to theright-turn waiting area is determined as the behavior of the hostvehicle (S75).

If another vehicle exists there, it is determined whether it is in theright-turn waiting area, or in or near the entrance to the opposing lanebased on “stopping vehicle position information D07.” If another vehicleis in the right-turn waiting area, and if a blind spot is formed at theentrance to the opposing lane by the vehicle, to move to the right-turnwaiting area is determined as the behavior of the host vehicle (S81). Ifanother vehicle is in the right-turn waiting area, and if a blind spotis formed behind the vehicle, to stop after moving at low speed isdetermined as the behavior of the host vehicle (S79). If another vehicleis at or near the entrance to the opposing lane, and if a blind spot isformed behind the vehicle, to stop after moving at low speed isdetermined as the behavior of the host vehicle (S79). If another vehicleis at or near the entrance to the opposing lane, if a bind spot isformed on the side road at the entrance to the opposing lane, to move tothe right-turn waiting area is determined as the behavior of the hostvehicle (S77).

Referring to FIG. 15, descriptions will be provided for the detailedprocedure for the knowledge tree (intersection inside) shown for stepS49 in FIG. 12. It is determined whether there exists another vehicle ator near the exit from the intersection based on the above-mentioned“intersection exit information D03.” If another vehicle exists there, tostop in the right-turn waiting area is determined as the behavior of thehost vehicle (S83). If no other vehicle exists there, it is determinedwhether there exists another vehicle at or near the entrance to theopposing lane based on “opposing lane entrance information S27.” Ifanother vehicle exists there, it is determined whether the area behindthe vehicle is visible based on “blind spot information D13.” If thearea behind the vehicle is visible, to perform collision avoidancecontrol is determined as the behavior of the host vehicle (S87). If thearea behind the vehicle is not visible, to stop in the right-turnwaiting area is determined as the behavior of the host vehicle (S85).

If no other vehicle exists at or near the entrance to the opposing lane,it is determined whether there exists a motorcycle which is going toturn right from inward of the back of the host vehicle based on“entanglement information D15.” If no motorcycle exists there, to moveto the front of the pedestrian crossing is determined as the behavior ofthe host vehicle (S89). If a motorcycle exists there, it is determinedwhether the front of the motorcycle is visible based on “blind spotinformation D17.” If the front of the motorcycle is visible, to move tothe front of the pedestrian crossing after letting the motorcycleovertake the host vehicle is determined as the behavior of the hostvehicle (S93). If the front of the motorcycle is not visible, to stop inthe right-turn waiting area is determined as the behavior of the hostvehicle (S91).

Referring to FIG. 16, descriptions will be provided for the detailedprocedure for the knowledge tree (intersection exit) shown for step S51in FIG. 12. It is determined whether there exists another vehicle on theopposing lane which is going to turn left at the intersection, orwhether there exists a motorcycle (another vehicle) which is going toturn right at the intersection from inward of the back of the hostvehicle, based on “left-turn-from-opposing-lane information D19,” or“entanglement-at-intersection information D21.” If another vehicleexists there, to move at low speed while letting the vehicle go beforethe host vehicle is determined as the behavior of the host vehicle(S95). If no other vehicle exists there, it is determined whether thereexists a pedestrian on the pedestrian crossing based on “pedestriancrossing information D23.” If a pedestrian exists there, to stop infront of the pedestrian crossing is determined as the behavior of thehost vehicle (S97). If no pedestrian exists there, it is determinedwhether there exists a pedestrian near the pedestrian crossing based on“pedestrian crossing exit/entrance information D25.” If a pedestrianexists there, to stop one second longer and pass through the pedestriancrossing if the pedestrian is not moving is determined as the behaviorof the host vehicle (S101). If no pedestrian exists there, to passthrough the pedestrian crossing is determined as the behavior of thehost vehicle (S99).

As discussed above, the following operation and effects can be obtainedfrom the second embodiment of the present invention.

Referring to the knowledge database 17, the obstacle determination unit26 determines whether there exists an obstacle in the obstacle detectionframes 42 using the knowledge trees (FIGS. 14 to 16) corresponding tothe position of the host vehicle. This makes it possible to calculate anappropriate degree of risk depending on the position of the host vehicleat each specific spot, and thereby to determine an appropriate vehiclebehavior.

Although the embodiments of the present invention have been describedabove, the descriptions or drawings constituting part of this disclosureshould not be understood as limiting the present invention. From thedisclosure, various alternative embodiments, examples and operationtechniques will be apparent to those skilled in the art.

REFERENCE SIGNS LIST

-   1 a, 1 b driving assistance apparatus-   2 obstacle detection frame-   12 map database-   16 degree-of-risk database-   17 knowledge database-   21 scene understanding unit (scene understanding device)-   23 map obtaining unit-   24 route calculator (route obtaining unit)-   25 detection frame selector-   26 obstacle determination unit-   27 degree-of-risk calculator-   28 blind spot overlap determination unit-   29 encoding processor-   30 degree-of-risk data obtaining unit-   31 knowledge tree obtaining unit-   42, 42′ obstacle detection frame-   48, 48′ close observation frame-   49 obstacle-   50 blind spot-   46 host vehicle-   51 scheduled traveling route-   52 bike (obstacle)-   53 pedestrian (obstacle)

The invention claimed is:
 1. A scene understanding device thatdetermines a degree of risk at a specific spot where a vehicle wouldotherwise bump into a vehicle or a pedestrian caused by crossed roads,comprising a computer programmed to: obtain map data, on a scheduledtraveling route, of a plurality of obstacle detection frames fordetecting an obstacle, the plurality of obstacle detection frames setfor both of the crossed roads in advance; obtain route data on thescheduled traveling route of a host vehicle; select a part of theobstacle detection frames to be determined depending on the scheduledtraveling route from the plurality of obstacle detection frames set forthe specific spot in advance; determine whether there exists an obstaclein the selected part of the obstacle detection frames; and calculate thedegree of risk at the specific spot based on a result of determiningwhether an obstacle exists there.
 2. The scene understanding deviceaccording to claim 1, wherein the computer is further programmed to:obtain as additional map data a close observation frame which closelyobserves the specific spot, the close observation frame set for thespecific spot in advance, determine whether a blind spot caused by theobstacle overlaps the close observation frame, and calculate the degreeof risk at the specific spot based on a result of determining whetherthe blind spot overlaps the close observation frame.
 3. The sceneunderstanding device according to claim 1, wherein the plurality ofobstacle detection frames is set for the specific spot, and the computeris further programmed to calculate the degree of risk at the specificspot based on a combination of obstacle detection frames including theobstacles, wherein the combination of obstacle detection framesincluding the obstacles is from the selected part of the obstacledetection frames.
 4. The scene understanding device according to claim3, further comprising: a degree-of-risk database where the combinationof obstacle detection frames including the obstacles is encoded, thecomputer programmed to store a relationship between the encodedcombination and the degree of risk in the degree-of-risk database; andan encoding processor that encodes the combination of obstacle detectionframes including the obstacles, wherein the computer is furtherprogrammed to calculate the degree of risk at the specific spot from theencoded combination using the degree-of-risk database.
 5. The sceneunderstanding device according to claim 4, wherein the computer isfurther programmed to detect attributes of the obstacles included in thecombination of obstacle detection frames including the obstacles at thespecific spot on the scheduled traveling route, and the encodingprocessor encodes the combination of obstacle detection frames includingthe obstacles with the attributes of the obstacles.
 6. The sceneunderstanding device according to claim 1, further comprising: aknowledge database that stores the plurality of obstacle detectionframes to be determined depending on a position of the host vehicle atthe specific spot, and an order of the plurality of obstacle detectionframes to be cautious about; and a sensor that detects the position ofthe host vehicle, wherein referring to the knowledge database, thecomputer is programmed to determine whether there exists an obstacle inthe selected part of the obstacle detection frames sequentially, basedon the position of the host vehicle.
 7. The scene understanding deviceaccording to claim 1, wherein the specific spot is an intersection wherethree or more roads meet, and the plurality of obstacle detection framesare set for an entrance to, an exit from, and an inside of, theintersection, and a pedestrian crossing.